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Abstract 


Although about 77% of ACT-tested high school graduates express an intention of earning a 
bachelor’s degree or higher (ACT, 2016a), the percentage of students who achieve academic 
success in their first-year courses, persist in college, and ultimately earn a bachelor’s degree 
within six years is substantially lower. Approximately 60% of students who initially enroll at 

a four-year institution graduate with a bachelor’s degree, and only 30% of students who 
initially enroll at a two-year institution earn an associate’s degree or certificate from their initial 
institution of attendance within six years (Kena et al., 2016). Alack of academic preparation 

in English and reading as evidenced by a significant percentage of incoming college students 
being required to take remedial coursework in these subject areas has been offered as one 
explanation for the low college graduation rates. 


The purpose of this research was to develop a preliminary English Language Arts (ELA) 
Readiness Benchmark to provide prospective students with information on the level of 
knowledge and skills needed to have a reasonable chance of success in ELA-related first-year 
courses. This is a preliminary study on ELA readiness as the data used to derive the 

ELA Benchmark was based on earlier freshman cohorts who took the former ACT® writing test; 
that is, ELA scores for these students were estimated based on their English, reading, and 
concorded writing scores (empirical evidence supporting this approach and more information 
on the former and current ACT writing tests are provided in Appendix A). 


This research had three components: Study 1 identified the English and social science courses 
that college students take most often in the first year of college, overall and by general major 
categories. In English, the most prevalent course was English Composition |. In the social 
sciences, multiple courses were identified as typically taken by students: American History, 
Other History, Psychology, Sociology, Political Science, and Economics. These patterns were 
seen irrespective of students’ general major categories. These are the same courses that were 
used to derive the ACT College Readiness Benchmarks in English and reading, respectively. 


Study 2 derived a preliminary empirically-based ELA Readiness Benchmark in English and 
social science courses by identifying the ACT ELA score associated with a 50% probability 

of earning a grade of B or higher in the seven ELA courses identified from Study 1. When 
combining grade data for these seven courses into a single course success model that 
included an indicator for content area (English vs. the social sciences), the typical ACT ELA 
score that was associated with at least a 50% chance of earning a B grade or higher in ELA- 
related courses was determined to be 20. Moreover, this cutoff score was also associated with 
an approximate 75% chance of earning a C grade or higher. 


Study 3 validated the ELA Readiness Benchmark on more distal indicators of success. Results 
demonstrated that students who met the ELA Readiness Benchmark were more likely to earn 
a cumulative grade point average (GPA) of 3.0 or higher over time, persist in college, and 
ultimately earn a degree. Providing prospective students with ELA readiness information based 
on their English, reading, and writing skills may help facilitate the transition to college by raising 
their awareness of the literacy skills that are required to meet the demands of the array of ELA- 
related courses they will face in college. Such feedback can send a signal to students as to the 
level of readiness that is needed to avoid having to take remedial coursework in English and 
reading that can impede students’ progress towards earning a college degree. 


Development and Validation of a Preliminary ELA 
Readiness Benchmark based on the ACT ELA Score 


English Language Arts (ELA), the integration of reading, writing, and speaking skills, has 
become increasingly important as the United States aims to develop students with strong and 
adaptable skills that can be applied to a rapidly changing world economy. The Common Core 
State Standards for English Language Arts & Literacy in History/Social Studies, Science, and 
Technical Subjects (Common Core State Standards Initiative (CCSSI), 2010) exemplifies 

the broad consensus that the nation needs students with integrated literacy skills that can 

be drawn upon across an array of academic subjects and career areas. In recognition of the 
importance of these integrated skills in both academics and the workplace, ACT has developed 
a Holistic Framework that includes core academic skills, such as ELA, as well as cross- 
cutting capabilities, behavioral skills, and educational and career navigation skills considered 
important for college and career readiness (Camara, O’Connor, Mattern, & Hanson, 2015; 
Mattern et al., 2014). According to this framework, to attain the necessary literacy skills for 
success, “students must learn to engage with complex print and digital texts written for a 
variety of purposes, think critically about what they read and hear, articulate their thoughts in 
a range of spoken and written genres, and communicate collaboratively with others” (Camara 
et al., 2015; p. 13). Lacking adequate literacy proficiency, students will be unable to handle 
the quantity and complexity of reading tasks required in college. Similarly, they are likely to 
struggle as they transition into the workforce (Schoenbach, Greenleaf, & Murphy, 2012). 


The increased emphasis on ELA skills for college and career success can be traced to 
research findings on the costs associated with low levels of academic achievement of high 
school graduates transitioning to college and the workforce. In particular, A Nation at Risk 
(Gardner, Larsen, Baker, Campbell, & Crosby, 1983) identified a lowering of academic 
standards in K-12 that ultimately led to an excessively high number of high school graduates 
requiring remediation once in college. Unfortunately, additional studies over the ensuing 
decades have continued to find that too many high school graduates in the United States find 
themselves underprepared for college-level coursework. In the 2011—2012 school year, nearly 
one-third of first- and second-year undergraduate students at public postsecondary institutions 
reported taking at least one remedial course (Skomsvold, 2014), slightly higher than the 
percentages reported in previous NCES reports (Lewis & Farris, 1996; Mansfield & Farris, 
1991; Parsad & Lewis, 2003). Remediation rates based on actual transcripts, however, are 
even higher. More than 39% of students who initially enrolled at 4-year public institutions and 
68% of students who initially enrolled at 2-year public institutions in 2003-04 took at least one 
remedial course (Radford & Horn, 2012). Focusing specifically on ELA-related courses, nearly 
11% of the students at 4-year institutions and 28% of the students at 2-year institutions took a 
remedial English or reading course (Radford & Horn, 2012). 


Students do not generally receive college credits for remedial courses, and a recent 
longitudinal NCES study that tracked students’ college experience over six years found 

that students who did not take any remedial English or reading courses were more likely to 
complete credit-bearing English courses than those who took the remedial courses (Chen, 
2016). Being placed into first-year, credit-bearing courses in college generally depends on 
students’ performance on admission tests and/or placement tests, as well as high school 
grades and the strength of their high school curriculum. Research on the relationship between 
ACT scores and academic performance in college has found that students with higher 
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ACT scores tend to earn higher grades in individual courses, including English Composition 
and reading-intensive courses (Allen, 2013; Allen & Sconing, 2005; Westrick, 2016). 
Furthermore, academic preparedness for college as measured by standardized test scores 
such as the ACT® and high school coursework and grades is positively related to college 
success indicators such as first-year grade point average (FYGPA), academic persistence, 
and, ultimately, degree completion (Radunzel & Noble, 2012; Westrick, Le, Robbins, Radunzel, 
& Schmidt, 2015). Similarly, students who were not required to take remedial English/reading 
courses were also more likely to persist and complete a bachelor’s degree at four-year public 
institutions regardless of whether students started at two-year or four-year public institutions 
(Chen, 2016). 


The ACT ELA Score 


To provide students with an aggregate measure of their readiness in English, reading, and 
writing, ACT introduced the ACT ELA score in fall 2015 for ACT-tested students who take the 
optional ACT writing test. The ACT ELA score is a combination of the ACT English, reading, 
and writing scores; it ranges between 1 and 36. 


The optional ACT writing test was first introduced in 2005, and in fall 2015, a number of 
enhancements from the former version were introduced. The enhancements included 
redesigning the writing prompts, extending the testing time from 30 minutes to 40 minutes, and 
using an analytical rubric on four writing domains for scoring instead of using a holistic six-point 
rubric. To help with transitioning from the former to the current ACT writing test, a concordance 
table between the former and the current writing scores was developed, allowing researchers 
to estimate ELA scores for graduating cohorts prior to 2015 (ACT, 2015). For a more detailed 
discussion of differences between the former and current ACT writing tests, and empirical 
evidence supporting the use of the concordance table for estimating ELA scores on earlier 
cohorts for specific purposes, refer to Appendix A. 


Many states and independent school districts see the value in providing students with an 
aggregate achievement measure that captures combined skills measured by the English, 
reading, and writing tests, such as the ACT ELA score. Currently, 11 of the 16 states that 
administer the ACT statewide include the optional ACT writing test, and roughly one-half of the 
independent school districts that offer the ACT to their high school students include the 

ACT writing test. For accountability purposes, states are now requesting information about the 
ACT ELA scores that are associated with a reasonable chance of graduating from high school 
with the literacy skills that are needed to be successful in college. 


Readiness Benchmarks 


To provide students with information about their readiness for first-year college courses, 

ACT developed empirically-derived benchmarks to identify students who had a high likelinood 
of being successful once in college based on their performance on the ACT (Allen, 2013; Allen 
& Sconing, 2005). ACT derived subject-specific benchmarks by estimating the ACT subject 
score associated with a 50% probability of earning a grade of B or higher in the typical credit- 
bearing, first-year course completed by students in the respective subject matter. For example, 
the ACT English Benchmark of 18 was derived based on the relationship between ACT English 
scores and course grades in English Composition |; for the other three subject areas, the 


courses examined were College Algebra, Biology, and social science courses for the 
ACT mathematics, science, and reading tests, respectively.' 2 


Recently, ACT developed the ACT STEM (Science, Technology, Engineering, and 
Mathematics) Benchmark to provide students with an indication of their readiness for 
mathematics and science courses typically taken during the first year by students entering 
STEM disciplines. These courses included Calculus, Chemistry, Biology, Physics, and 
Engineering (Mattern, Radunzel, & Westrick, 2015) and differed from those used to develop 
the ACT College Readiness Benchmarks in mathematics (College Algebra) and science 
(Biology). The ACT STEM Readiness Benchmark based on the ACT STEM score was found to 
be a 26 (Radunzel, Mattern, Crouse, & Westrick, 2015).° 


Given the importance of integrated literacy skills for academic and workplace success, the 
focus of the current study was to explore ELA readiness and what that means in relation to 
being successful in first-year ELA-related courses that include English and social science 
courses. Similar to establishing the ACT STEM Benchmark, we first wanted to explore whether 
there was empirical evidence to suggest that an ELA Readiness Benchmark should be tied to 
specific majors and therefore informed strictly by the typical first-year courses taken by ELA- 
related majors. This approach would be supported if students majoring in ELA-related fields 

of study are taking substantially different English and social science courses than the typical 
first-year student. 


If this was not found to be the case, we would then take an approach different from that used 
in the ACT STEM Benchmark study. Rather than developing an ELA Readiness Benchmark 
specifically based on the academic requirements of ELA-related majors, we would develop 

an ACT ELA Readiness Benchmark based on the ACT ELA score associated with a high 
probability of succeeding in the most common first-year ELA-related courses taken by students 
across all academic majors (this latter scenario was suggested by Study 1 results). 


Providing students with an indicator of their ELA readiness has a practical benefit. If provided 
early enough, students who intend to attend college but are not on track to be ready for 
relevant college-level coursework can take preemptive action to increase their ELA readiness, 
and thereby reduce their chances of needing to take remedial coursework in English and 
reading. 


Another purpose of establishing an ELA Readiness Benchmark for the ACT assessment is 
to develop ACT Aspire ELA Benchmarks for earlier grades by linking early-grade test scores 


The ACT College Readiness Benchmarks were updated in 2013 utilizing more current data. The new analyses revealed 
no change in the ACT English and Mathematics Benchmarks of 18 and 22, respectively. The ACT Reading Benchmark 
increased from 21 to 22, and the ACT Science Benchmark decreased from 24 to 23 (Allen, 2013). 


Two years after ACT’s College Readiness Benchmarks were announced, the College Board followed with their own 
benchmarks for the SAT that were based on first-year grade point average (Kobrin, 2007; Wyatt, Kobrin, Wiley, Camara, 
& Proestler, 2011). Later, they updated their benchmarks to be based on success in first-year college courses (College 
Board, 2017). More recently, two multistate consortia, the Partnership for Assessment of Readiness for College and 
Careers (PARCC) and Smarter Balanced, have developed new assessments aligned to the Common Core State 
Standards (CCSS). PARCC has their own “college and career ready” cut-scores based on earning a C or better average 
in college (n = 476; Nichols-Barrer, Place, Dillon, & Gill, 2015, p.16) for ELA and mathematics. Smarter Balanced has 
achievement-level descriptors for ELA and mathematics (Smarter Balanced Assessment Consortium, 2013). 


Ny 


wo 


The study by Mattern et al. (2015) related the ACT mathematics score to course success in Calculus and the 

ACT science score to course success in the combined first-year science courses of Chemistry, Biology, Physics, and 
Engineering. The study by Radunzel et al. (2015) developed an ACT STEM Benchmark based on the ACT STEM score 
(the rounded average of the ACT mathematics and science scores) using a combined model that included an indicator 
for content area (mathematics vs. science) and was based on the five mathematics and science courses identified as 
being typically taken by STEM majors. 
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to the ACT ELA Benchmark. In 2014, ACT launched ACT Aspire®, a battery of assessments 
which assess students’ mastery of math, ELA, and science in grades 3 through 10.4 Readiness 
benchmarks have been developed for ACT Aspire that indicate whether students are on target 
to meet the ACT College Readiness Benchmarks in grade 11,° allowing for the articulation of 
what students need to know and be able and willing to do at key transition points along the 
K-Career continuum. Such information helps students and teachers know if a student is on 
track for college and career readiness. An ELA Readiness Benchmark for early grades will help 
educators identify students who are at risk of not having the literacy skills that are needed to 
be successful in college by the time they complete high school. 


Current Study 


The purpose of the current study was to develop a preliminary indicator or benchmark of ELA 
readiness based on the ACT ELA score. To accomplish this, three studies were conducted. The 
goal of Study 1 was to determine the typical first English and social science courses taken by 
students during their first year of college, overall and by general major categories. Building on 
Study 1, Study 2 empirically derived an ELA Readiness Benchmark based on the relationship 
between the ACT ELA score and performance in the identified typical first-year courses taken 
in English and the social sciences. Finally, Study 3 was conducted to examine the validity 

of the ELA Readiness Benchmark for predicting other college outcomes including earning a 
cumulative GPA of 3.0 or higher, persisting in college, and completing a college degree. 


Study 1: Course Taking Patterns in English 
and the Social Sciences 


Sample 

Data were available on first-year courses at 59 two-year and 44 four-year postsecondary 
institutions from four states for a total of 103 institutions. All but two of the institutions are public 
institutions, with the other two being private institutions. Of the 44 four-year institutions, 

10 have highly selective or selective admissions policies. The remaining 34 institutions have 
traditional, open, or liberal admissions policies. All of the two-year institutions have open 
admissions policies.* The median average ACT Composite score across institutions was 

19.7 (1% Quartile = 18.8; 3° Quartile = 21.2).” Course information was available for 397,861 
ACT-tested students who had enrolled in college as first-time entering students from the 2006 
through 2014 freshman cohorts.® 


s 


Since the introduction of the ACT Aspire assessments, students who completed the ACT Aspire English, reading, and 
writing assessments have received an ELA Composite score. 


a 


Benchmarks had previously been developed for ACT Explore® (grades 8 and 9) and ACT Plan® (grade 10). 


2 


Admission policy was reported by institutions according to the high school class ranks of their accepted freshmen: the 
majority of freshmen at highly selective schools are in the top 10%, selective in the top 25%, traditional in the top 50%, 
and liberal in the top 75% of their high school class. Institutions with open admissions policies accept all high school 
graduates to limit of capacity. In this study, four-year institutions were classified as more selective (selective or highly 
selective) or less selective. 


mT 


The median average ACT Composite score across institutions was 21.6 for the four-year institutions and 19.1 for the 
two-year institutions. 


co 


Seventy-two percent of the students were enrolled at a four-year institution and 28% were enrolled at a two-year 
institution. 


Course Content Coding. Partnering institutions provided ACT with course transcript data that 
included such information as the grades earned, course titles, and course content codes.° For 
courses that were not assigned a content code by the participating institution, ACT research 
staff assigned a code based on the course title and description from the institution’s course 
catalog. We then identified for each student the first English and social science courses taken 
in the first year. Based on this coding, 344,518 students were identified as taking an English 
course in their first year. Of these students, 12% took more than one English course in the 
same semester. Essentially, these students had multiple “first” English courses in college, and 
as such, all of the “first” English courses taken were included in the Study 1 analyses. For the 
social sciences, 355,869 students were identified as taking a social science course in their 
first year. Students were much more likely to take more than one social science course in the 
same semester—36%. As was the case with the English results, all of the “first” social science 
courses were included in the analyses. 


Coding General Major Categories. Institutions also provided students’ declared majors over 
time by reporting a six-digit Classification of Instruction Program (CIP) code for each term 
enrolled for most students. The first-year CIP codes were used to classify students into the 
following seven general major categories: Arts & Humanities, Social & Behavioral Sciences, 
General Studies, Life Sciences, Science & Engineering, Career Related, and Other Non- 
Science Related Majors such as Education and Business. These general major categories 
are similar to those used by the National Academies of Sciences, Engineering, and Medicine 
taxonomy of fields (The National Academies of Science, Engineering, and Medicine, 2006). 
The first two digits of the CIP codes were used to classify students into one of the general 
major categories if their declared major from both the fall and spring terms of year one came 
from the same general category (see Appendix B, Table B1). There was interest in examining 
whether the typical first-year courses taken in English and the social sciences differed 

across major categories, with a primary focus on comparing ELA-related major categories 
such as Arts & Humanities, Social & Behavioral Sciences, and General Studies to other 
major categories such as Life Sciences and Science & Engineering. Using the course grade 
information, the most prevalent first English and social science courses taken in the first-year 
for each general major category were identified. Characteristics of the students included in the 
English and social science samples (overall and by major category) are summarized in 
Tables 1 and 2, respectively. 


® Acommon course content code list was developed by ACT and used to identify similar content-related courses across 
institutions, e.g., English Composition I, Psychology, etc. 
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Table 1. Summary of Student Characteristics of English Sample for Study 1 


Other 
Social & Non-Science 
All Arts & Behavioral General Life Science & Career Related 
Characteristics Students Humanities Sciences Studies Sciences Engineering Related Majors 
N (Students) 344,518 12,277 24,285 60,378 47,503 22,530 8,064 50,017 
Gender 
Female 53.5% 56.4% 58.6% 53.5% 71.8% 21.6% 20.9% 52.8% 
Male 45.8% 42.7% 40.7% 45.5% 27.6% 77.7% 78.4% 46.4% 
Missing 0.8% 0.9% 0.7% 1.0% 0.6% 0.7% 0.7% 0.8% 
Race/Ethnicity 
African American 12.9% 10.4% 15.4% 11.0% 11.7% 8.2% 9.8% 12.3% 
Asian American 1.9% 1.3% 1.2% 2.0% 2.3% 3.5% 1.9% 1.6% 
Hispanic 6.6% 5.9% 9.9% 4.9% 7TA% 9.7% 5.6% 5.5% 
Other 7.2% 6.7% 7.1% 9.3% 7TA% 5.3% 6.7% 6.6% 
White 67.0% 70.5% 62.0% 68.2% 67.4% 68.2% 71.2% 69.8% 
Missing 44% 5.2% 4.5% 4.6% 3.9% 5.0% 4.7% 4.2% 
ACT Composite Score 
Mean 20.8 22.3 21.0 20.1 20.8 23.7 20.0 21.0 
Standard Deviation 44 45 44 41 4.3 48 4.3 44 
HSGPA 
Mean 3.25 3.29 3.25 3.19 3.36 3.48 3.13 3.30 
Standard Deviation 0.56 0.54 0.54 0.57 0.52 0.50 0.58 0.54 
Missing 14.2% 14.2% 13.6% 16.0% 11.1% 12.5% 17.1% 13.7% 


Note. Percentages may not sum to 100% due to rounding. The number of students in the All Students column is not 

the sum across the general major category columns since results are not shown for students who changed their majors 
during the first year (n = 33,014), those with undeclared majors during at least the fall term of their first year (n = 35,510), 
those who were not enrolled during the spring term of their first year (n = 37,316), and those for whom institutions did not 
provide students’ major information (n = 13,624). 


Table 2. Summary of Student Characteristics of Social Science Sample for Study 1 


Other 
Social & Non-Science 
All Arts & Behavioral General Life Science & Career Related 
Characteristics Students Humanities Sciences Studies Sciences Engineering Related Majors 
N (Students) 355,869 11,684 27,221 62,584 51,543 23,133 6,990 52,749 
Gender 
Female 54.5% 57.3% 59.5% 53.9% 72.3% 23.7% 24.2% 53.3% 
Male 44.6% 41.6% 39.7% 45.1% 27.1% 75.3% 74.9% 45.8% 
Missing 0.9% 1.1% 0.8% 1.0% 0.6% 1.0% 0.9% 0.9% 
Race/Ethnicity 
African American 12.1% 9.5% 14.2% 10.4% 10.8% 7.7% 9.8% 11.4% 
Asian American 2.0% 1.3% 1.3% 2.1% 2.6% 3.6% 2.1% 1.8% 
Hispanic 6.8% 6.1% 10.0% 4.9% 7.7% 10.3% 5.9% 5.7% 
Other 7.1% 6.6% 6.8% 9.3% 7.1% 5.7% 6.8% 6.5% 
White 67.5% 71.2% 62.8% 68.7% 67.8% 67.4% 70.2% 70.3% 
Missing 4.5% 5.4% 4.8% 4.6% 4.0% 5.4% 5.2% 44% 
ACT Composite Score 
Mean 21.2 22.9 21.4 20.4 21.3 24.2 20.5 21.4 
Standard Deviation 4.6 47 4.6 4.2 4.5 4.9 4.6 45 
HSGPA 
Mean 3.30 3.34 3.30 3.22 3.41 3.53 3.20 3.34 
Standard Deviation 0.56 0.53 0.53 0.56 0.51 0.48 0.52 0.53 
Missing 13.9% 14.4% 13.3% 15.8% 10.8% 12.3% 16.5% 13.5% 


Note. Percentages may not sum to 100% due to rounding. The number of students in the All Students column is not 

the sum across the general major category columns since results are not shown for students who changed their majors 
during the first year (n = 35,448), those with undeclared majors during at least the fall term of their first year (n = 37,354), 
those who were not enrolled during the spring term of their first year (n = 34,311), and those for whom institutions did not 
provide students’ major information (n = 12,852). 


Results 


The percentage of students who took an English and social science course by content area 


was computed for the total sample and by major category."° The results for English and the 


social sciences are summarized in Tables 3 and 4, respectively." 


English Results. Across all students in the sample irrespective of their major, English 


Composition | was the most common first course (68%) taken in English. This finding also 


10 Weighted percentages based on institution representation are reported. The weights were applied at the institution level 
and were determined by comparing the distribution of the institutions included in the sample to those included in the 
ACT-tested college-enrolled population with respect to institution type and admission selectivity (two-year vs. four-year/ 
less selective vs. four-year/more selective). The selectivity of institution’s admission policies were self-reported by the 
institutions using five levels that classified their level according to the typical ACT Composite score and high school 
ranks of their accepted freshmen. Specifically, the population included a larger proportion of more selective four-year 
institutions and fewer two-year institutions (26% four-year/more selective, 39% four-year/less selective, vs. 35% two- 
year) as compared to the sample (10% four-year/more selective, 33% four-year/less selective, vs. 57% two-year). See 
Tables 5 and 6 for more details about the population. 


"The same general conclusions for the overall samples concerning the typical first English and social science courses 
taken emerged when the distributions were evaluated (1) by institution type or (2) among students who had taken the 
former ACT writing test so that an ELA score could be computed. 
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held within each of the major categories examined with the percentage of students taking 
English Composition | as their first English course ranging from 63% for those majoring 

in Arts & Humanities to 75% for those majoring in Science & Engineering.'? Moreover, an 
additional 13% of all students first enrolled in a Developmental English course, a course that 
is often considered a non-credit bearing course. These results coincide with the ACT College 
Readiness Benchmark research that derived the ACT English Benchmark using course grades 
in English Composition | (Allen, 2013; Allen & Sconing, 2005). Even when focusing exclusively 
on students majoring in English Language and Literature/Letters (N=2,294; a subset of Arts 

& Humanities; data not shown in table), the typical first English course taken was English 
Composition | (43%) with an additional 5% taking a Developmental English course as their first 
English course." 


Table 3. Distribution (%) of Students’ First College English Course by Major Category 


Social & Other 
All Arts & Behavioral General Life Science & Career Non-Science 
English Content Area Students Humanities Sciences Studies Sciences Engineering Related Related Majors 
Developmental English 12.5 74 10.0 13.9 11.1 5.4 15.2 11.1 
ESL English 0.1 0.0 0.0 0.0 0.1 0.1 0.3 0.1 
English Composition | 67.7 63.4 65.6 72.7 68.0 75.2 67.0 67.5 
English Composition II 6.9 8.9 7.1 6.2 8.3 9.9 6.0 6.2 
English Literature 5.1 11.6 5.5 2.5 5.4 4.0 1.2 4.5 
Speech 15.4 12.4 15.5 12.6 13.0 7.5 11.4 19.6 
Other Writing Intensive 1.3 2.7 2.9 1.1 0.6 1.1 3.4 2.3 
Other English 3.4 5.9 4.5 3.2 2.5 2.6 3.1 47 
Multiple courses 12.2 12.2 11.1 11.0 8.6 4.9 7.2 15.3 


Note. Data are based on 103 two- and four-year postsecondary institutions from four states (2006 through 2014 freshman 
cohorts). There were 344,518 total students. See Table 1 for the number of students by general major category. Examples 
of courses included in Other Writing Intensive courses include Technical Writing and Creative Writing. Examples included 
in the Other English category include Introduction to Poetry, Introduction to Shakespeare, and Introduction to Fiction. 
Percentages will sum to above 100% due to the fact that some students took multiple English courses in the same term. 


This finding held even among students who were not categorized into one of the seven major categories presented 
in Table 3 (English Composition | at 65%); see footnote to Table 1 for reasons why these 119,464 students were not 
categorized into one of the seven major categories. 


"8 Students were identified as majoring in English Language and Literature/Letters if the first two-digits of their CIP major 
code was 23 in both the fall and spring terms of their first year. 


Table 4. Distribution (%) of Students’ First College Social Science Course by Major 


Category 
Social & Other 
All Arts & Behavioral General Life Science & Career Non-Science 
English Content Area Students Humanities Sciences Studies Sciences Engineering Related Related Majors 
Developmental Reading 6.6 3.7 4.3 7.5 5.2 2.8 8.9 5.3 
General Humanities 2.5 3.2 2.4 3.2 2.6 2.1 1.3 2.4 
American History 21.8 21.1 17.5 25.1 21.6 23.8 24.7 21.4 
Other History 10.3 17.9 9.6 7.7 6.9 8.8 5.8 9.3 
Psychology 31.6 23.2 37.5 32.0 42.0 21.2 23.1 27.5 
Sociology 15.9 13.1 19.7 15.1 17.7 10.0 12.2 14.8 
Geography 5.5 5.2 4.6 7.0 3.4 6.0 9.0 74 
Anthropology 3.8 7.9 6.3 2.3 3.2 3.5 2.3 3.7 
Archaeology 0.2 0.6 0.5 0.2 0.2 0.4 0.3 0.2 
Political Science 16.9 15.9 21.5 19.4 13.9 21.1 21.5 15.4 
Economics 8.4 3.2 5.3 6.1 3.2 10.2 5.6 20.6 
Law 2.4 0.6 5.5 0.6 0.3 0.8 0.4 41 
Philosophy/Logic 8.2 13.7 12.0 9.2 6.7 8.4 5.5 5.9 
Religion 1.7 2.7 2.5 1.8 1.4 1.4 1.1 1.3 
Ethics 1.3 1.1 1.6 1.3 1:5 1.9 3.2 1.2 
Human Growth & Development 0.4 0.1 0.2 0.2 1.0 0.0 0.0 0.6 
Other Social Science 4.0 4.6 5.4 3.8 4.0 4.0 2.6 3.5 
Multiple courses 35.9 32.8 45.7 38.8 29.2 22.2 25.3 39.4 


Note. Data are based on 103 two- and four-year postsecondary institutions from four states (2006 through 2014 
freshman cohorts). There were 355,869 total students, See table 2 for the number of students by general major category. 
Examples of courses included in the Other History course included World History and Western Civilization. Examples of 
courses included in Other Social Science courses include Introduction to Social Work, Mythology, and Social Problems. 
Percentages will sum to above 100% due to the fact that some students took multiple social science courses in the same 
term. 


Social Science Results. Across all students in the sample irrespective of their major, the 
most prevalent social science courses taken during their first year included Psychology (32%), 
American History (22%), Political Science (17%), Sociology (16%), Other History (10%), and 
Economics (8%). These same courses tended to be the most prevalent social science courses 
within each of the major categories examined (Table 4).'* These results coincide with the 

ACT College Readiness Benchmark research that derived the ACT Reading Benchmark using 
course grades in these same courses (Allen, 2013; Allen & Sconing, 2005). 


In conclusion, the results from Study 1 did not find substantial differences across major 
categories in the typical first-year English and social science courses taken. Therefore, the 
results supported using the same English and social science courses to develop the 

ACT ELA Readiness Benchmark that were used to develop the individual ACT College 
Readiness Benchmarks in English and reading. 


‘4 The most prevalent social science courses taken for students who were not categorized into one of the seven major 
categories presented in Table 4 were Psychology (31%), American History (21%), Sociology (17%), Political Science 
(15%), Other History (14%), and Developmental Reading (9%); see footnote to Table 2 for reasons why these 
119,965 students were not categorized into one of the seven major categories. 
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Study 2: Development of an ELA Readiness 
Benchmark 


Sample 

Course grade data used in the current study were provided by both two- and four-year 
postsecondary institutions who participated in research services offered by ACT, including 
state partnerships, the Course Placement Service, and Prediction Service. Data included 

in the analyses were limited to students from the 2006 through 2014 freshman cohorts who 
took the former ACT writing test. The ACT writing test is optional; for reference, the former 
ACT writing test was taken by 62% of 2014 ACT-tested high school graduates during any 
test administration, whereas 53% of 2014 ACT-tested high school graduates took the writing 
test during their last test administration prior to enrolling in college (ACT, 2014). Using a 
concordance table, students’ ACT writing scores were converted to current ACT writing scores 
that are used to calculate the ACT ELA score (ACT, 2015; see Table 4 of the 2015 ACT report 
for the concordance table). Students’ ELA scores were calculated as the rounded average of 
the ACT English, reading, and concorded writing scores from the student’s latest test record 
when the student took the ACT with writing (see Appendix A for empirical evidence supporting 
the use of the concorded writing scores in calculating an ACT ELA score for earlier cohorts to 
be used in the development of a preliminary ACT ELA Benchmark). 


The specific courses included in this study were those that were identified as being the 

typical first-year courses in English and the social sciences from Study 1. These courses 
included English Composition |, Psychology, American History, Political Science, Sociology, 
Other History, and Economics, which coincide with those used to develop the ACT College 
Readiness Benchmarks in English and reading (Allen, 2013). Similar to Study 1, students’ first 
English and social science courses taken during the first year were used in the analyses. '® 
For students who were enrolled in multiple ELA-related courses during the same term, grade 
information for a single course was randomly selected for inclusion in the analyses. 


The number of institutions and students varied based on the data available by course content 
area; these numbers along with additional information on key institutional and student 
characteristics of the sample are summarized in Tables 5 and 6. Compared to the population 
of postsecondary institutions where 2015 ACT-tested high school graduates enrolled in the 

fall of the same year, the study sample included more public than private institutions and more 
institutions from the Midwest and Southwest regions as compared to being located in the East 
and West regions (Table 5). The study sample also included slightly fewer four-year institutions 
and highly selective/selective institutions than the reference population group. 


‘8 This criterion is also used in ACT’s Course Placement Research services to avoid having intervening coursework 
influence test score-course outcome relationships. 
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Table 5. Summary of Institutional Characteristics of Study 2 


English Combined Total 

Characteristic Composition! Social Science Sample’ Population 
# of institutions 200 154 233 1,955 
Type of institution 

Four-year 58% 57% 60% 65% 

Two-year 42% 43% 40% 35% 
Admissions Policy 

Highly selective, selective 12% 12% 12% 26% 

Traditional, open, liberal 88% 88% 88% 74% 
Sector 

Private 11% 6% 12% 41% 

Public 89% 94% 88% 59% 
Region 

East 13% 1% 12% 43% 

Midwest 33% 41% 36% 25% 

Southwest 44% 46% 42% 12% 

West 10% 6% 10% 20% 
Average ACT Composite score 

Median 20.2 20.3 20.4 20.5 

1st Quartile to 3rd Quartile 18.9 to 21.7 19.2 to 22.1 19.0to 21.9 18.9 to 23.0 


Note. Combined social science courses include American History (74 institutions), Other History (57 institutions), 
Psychology (121 institutions), Sociology (75 institutions), Political Science (60 institutions), and Economics 

(33 institutions). Fifty-eight of the institutions had grades available for only one social science course; 96 institutions 
had grades available for more than one social science course (25 institutions had 2 courses; 19 had 3 courses; 21 had 
4 courses; 15 had 5 courses; and 16 had all 6 courses). Population includes postsecondary institutions where 2015 
ACT-tested graduates initially enrolled in fall 2015 (determined using enrollment records from the National Student 
Clearinghouse). This graduating class was selected because it was the most recent cohort with enrollment information 
available at the time of these analyses. 


' The total number of institutions in the sample does not equal the sum of institutions across the two content areas since 
121 institutions had course grade data available in both content areas. 


Relative to the population of 2015 ACT-tested high school graduates, the study sample had 
fewer African American and Hispanic students (Table 6). Students in the study sample also 
tended to have slightly higher ACT Composite scores than the reference population, on 
average. To address some of these student and institutional differences between the sample 
and population, weights based on ACT Composite score, high school GPA (HSGPA), and race/ 
ethnicity were applied at the student level, while weights based on the student representation 
by institution type and institution selectivity were applied at the institution level. 
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Table 6. Summary of Student Characteristics of Study 2 


English Combined 

Characteristics Composition! socialscience Totalsample Population 
N (Students) 107,142 91,133 198,275 1,360,170 
Gender 

Female 53% 57% 55% 52% 

Male 46% 42% 44% 47% 

Missing 1% 1% 1% 1% 
Race/Ethnicity 

African American 11% 9% 10% 15% 

Asian American 2% 2% 2% 3% 

Hispanic 6% 9% T% 12% 

Other 3% 5% 4% 5% 

White 72% 70% 71% 58% 

Missing 5% 5% 5% 7% 
ACT Composite Score 

Mean 21.6 22.9 22.2 20.5 

Standard Deviation 3.8 4.7 4.3 5.3 
ACT ELA Score 

Mean 20.6 21.8 21.2 20.0 

Standard Deviation 3.9 4.7 4.3 6.0 
HSGPA 

Mean 3.26 3.42 3.33 3.17 

Standard Deviation 0.52 0.51 0.52 0.66 


Note. Population includes ACT-tested high school graduates of 2015 from states where at least 50% of students took the 
ACT. For the population, 50% of the students had taken the former ACT writing test. Combined social science courses 
include American History (14,576 students), Other History (8,025 students), Psychology (35,905 students), Sociology 
(12,842 students), Political Science (11,594 students), and Economics (8,191 students). 


Analyses 


The same methodology employed to derive the ACT STEM Benchmark was used in the 
current study (Radunzel et al., 2015). To determine the ACT ELA Benchmark, course success 
was defined as earning a grade of B or higher. We also examined students’ chances of 
earning a grade of C or higher. Course grade data for English Composition | and the social 
science courses were combined into a single course success model that included an indicator 
for content area (0 for English and 1 for social science courses). To account for students 
being nested within institution and course, hierarchical logistic regression models were run 

to estimate the relationship between ACT ELA scores and course success in ELA-related 
courses. Student weighting was employed when estimating the models to ensure that the 
sample was representative of a larger population of ACT-tested high school graduates in terms 
of race/ethnicity, ACT Composite score, and HSGPA."® The NLMIXED procedure within 


16 This is the population to which the ACT College Readiness Benchmarks are reported on, and thus, is the envisioned 
group to who the ELA Benchmark would be conveyed to provide an indicator of their readiness for typical, first-year, 
ELA-related courses. 


SAS 9.2 statistical software was used to fit the models and obtain the parameter estimates for 
each college and course combination. 


The estimated parameter estimates derived for each course within an institution was used to 
compute institution-specific probabilities of success as a function of ACT ELA scores." First, 
to combine the results across the multiple social science courses, the median probability of 
success at each ACT ELA score was computed across the available social science courses 
for each institution. Then, after applying institutional weighting, '® the weighted typical score- 
specific probabilities of success for each content area (English and the social sciences) were 
computed across institutions. Finally, giving equal weight to the two content areas, the typical 
score-specific probabilities of success across the content areas were computed. Based on 
these final typical probabilities of success, the ELA Benchmark score was determined as the 
score associated with at least a 50% chance of earning a grade of B or higher in an ELA- 
related course. 


Results 


Descriptive Results 


The distribution of course grades in English Composition | and each of the social science 
courses was examined. Overall, 52% of the students earned a grade of B or higher in the 
ELA-related courses. The percentage of students earning a B or higher was higher for English 
Composition | than for the combined social science courses (55% vs. 49%).'° 


Model-Based Results 


Figure 1 provides the typical chances of earning a grade of B or higher in first-year ELA-related 
courses based on combining grade data for English Composition | and multiple social science 
courses into a single course-success model.”° The lowest ACT ELA score associated with at 
least a 50% chance of earning a grade of B or higher in a typical ELA-related course is 20 and 
therefore is determined to be the ACT ELA Readiness Benchmark.”' As illustrated in the figure, 
a student’s chances of earning a grade of B or higher in ELA-related courses increase as their 
ACT ELA score increases. For example, students with an ACT ELA score of 27 or higher have 
greater than a 75% chance of earning a B or higher. The probability of earning a C or higher is 
also plotted in Figure 1. For students with an ACT ELA score of 20 or higher, their chances of 
earning a C or higher in a typical first-year ELA-related course are greater than 75%. 


'’ The fixed effect intercept and slope estimates associated with the content area indicator were incorporated into the 
estimates of the institution-specific intercepts, slopes, and probabilities for the social science courses. 


18 Institutional weighting consisted of applying weights to the institution specific cut scores to make the sample of students 
similar to an ACT-tested college enrollee population with respect to the type and selectivity of the institutions attended. 


'8 Across the six social science courses examined, roughly one-half (49%) of the students earned a B or higher, though 
this percentage did vary by course (from 41% in Economics to 54% in American History and Political Science). 


20 The parameter estimates for the single-course success model are provided in Table B2 of Appendix B. The estimated 
probabilities obtained from this model differ slightly from those presented in Figure 1. The reason for this is that the 
probabilities shown in Figure 1 incorporated both the student and institutional weighting; the institutional weighting was 
applied to determine the typical probabilities across institutions. In contrast, only the student weighting was incorporated 
to obtain the parameters estimates for the single-course success model. 


21 The median of the content-specific median cut scores derived from the single course-success model was also a 20 
(median Q1=18; median Q3 = 22). The model correctly classified whether a student would earn a B or higher in ELA- 
related courses for 61% of the cases, which is a 17% increase over chance. Of those who earned a B or higher in 
an ELA-related course, 72% were predicted to earn a B or higher. Of those who earned below a B in an ELA-related 
course, 49% were predicted to earn below a B. 
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Figure 1. The typical probability of success in ELA-related courses by ACT ELA 
score. The English related course was English Composition |. The social science 
courses included Psychology, American History, Political Science, Sociology, Other 
History, and Economics. 


In supplementary analyses, an ACT ELA cut score was also derived based on first computing 
the weighted median probabilities of success across institutions that were obtained from 
separate content-specific course success models. Then the typical probabilities of success 
were averaged across the two content areas and the score associated with at least a 50% 
chance of success was determined.” The separate models employed the same approach that 
was used to derive the individual ACT College Readiness Benchmarks (Allen, 2013), but they 
used the ACT ELA score instead of the individual ACT English and reading scores. Based 

on this alternative approach, the same ACT ELA cut score was obtained. Results from this 
alternative approach are not presented here. 


Study 3: Validation of ACT ELA Readiness 
Benchmark 


In Study 3, we examined whether students who met the ACT ELA Benchmark were more 
successful later in college than students who did not meet the Benchmark. Multiple indicators 
of later success in college were examined: Earning a cumulative GPA of 3.0 or higher in years 
one through four; persisting in college to the second, third, and fourth year; and graduating 
with a degree in four, five, or six years. 


22 The cut score of 20 was also obtained by taking the median of the two individual content-specific ELA cut scores that 
were identified to be associated with at least a 50% chance of earning a grade of B or higher in the corresponding 
content-related course(s). The individual content-specific ELA cut scores were calculated as the weighted median 
cut scores across institutions that were obtained from separate content-specific course success models (English 
Composition | as one and combined social science courses as the other). 


Sample 

Longitudinal college outcomes data used in the current study were provided by two and 
four-year postsecondary institutions who have participated in state partnerships and 

research services offered by ACT. Data included in the analyses were students from the 
2006 through 2009 cohorts who took the former ACT writing test.27 Student outcomes were 
tracked for at least four years at the initial institution attended, and, where possible, across 
in-state institutions.24 Two state systems provided data on all of their two- and four-year 
public institutions, representing 100% of the two-year student sample and 39% of the four- 
year student sample. Tracking across a state system is particularly relevant for the two-year 
sample given that many students beginning at two-year institutions transfer to a four-year 
institution without first receiving a credential from the two-year institution (Shapiro, Dundar, 
Ziskin, Chiang, Chen, Harrell, & Torres, 2013). Because students from the four-year sample 
were primarily tracked at the initial institution attended, while students from the two-year 
sample were tracked across in-state institutions, the definitions of the persistence and degree 
completion outcomes that were used in this study differed between the two samples. For 

the four-year sample, persistence was defined as remaining enrolled at the initial institution 
attended. In comparison, persistence for the two-year sample was defined as remaining 
enrolled at an in-state two- or four-year institution. For the four-year sample, bachelor’s 
degree completion from the initial institution attended was evaluated. For the two-year sample, 
completion of an associate’s or bachelor’s degree from an in-state institution was examined. 


College outcomes data were available for 69,498 students from 49 four-year institutions and 
5,967 students from 34 two-year institutions. Students’ ELA scores were calculated as the 
rounded average of the ACT English, reading, and concorded writing scores from the student's 
latest test record when the student took the ACT with writing. The sample for the study does 
not represent students and institutions nationally. A large majority of both the two- and four- 
year institutions came from the southwest region (Table 7). Additionally, about three-fourths of 
the four-year institutions and all of the two-year institutions were public institutions. Students 
from the four-year sample tended to be better prepared academically than those from the 
two-year sample (Table 8). For example, more than three-fourths (78%) of the four-year 
sample and fewer than one-half (45%) of the two-year sample met the ACT ELA Benchmark 
of 20. Additional information on key institutional and student characteristics of the sample is 
summarized in Tables 7 and 8. 


3 Forty-three percent of the full ACT-tested four-year sample of 163,366 students took the former ACT writing test. Only 
14% of the full ACT-tested two-year sample of 43,875 students took the former ACT writing test. 

4 For earlier cohorts, up to eight years of outcome data were available. For the most recent cohorts, at least four years of 
longitudinal data were available. For two state systems, students were tracked across in-state institutions. 
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Table 7. Summary of Institutional Characteristics of Study 3 


Characteristic Four-year institutions Two-year institutions 
# of four-year institutions 49 34 
Admissions Policy 
Highly selective, selective 24% 0% 
Traditional, open, liberal 76% 100% 
Sector 
Private 29% 0% 
Public 71% 100% 
Region 
East 12% 0% 
Midwest 10% 0% 
Southwest 73% 100% 
West 4% 0% 


Average ACT Composite score 


Median 21.5 18.6 
1st Quartile to 3rd Quartile 20.3 to 23.2 18.2 to 19.1 


Note. Percentages may not sum to 100% due to rounding. 
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Table 8. Summary of Student Characteristics of Study 3 


Characteristics Four-year institutions Two-year institutions 
N (Students) 69,498 5,967 
Gender 
Female 55% 58% 
Male 42% 41% 
Missing 4% 2% 
Race/Ethnicity 
African American 8% 12% 
Asian American 3% 2% 
Hispanic 11% 5% 
Other 3% 8% 
White 65% 66% 
Missing 9% 7% 
ACT ELA Benchmark attainment 
Not met 22% 55% 
Met 78% 45% 
Academic Performance Mean (SD) Mean (SD) 
ACT Scores 
Composite 23.6 (4.6) 19.5 (3.9) 
English 23.6 (5.6) 19.2 (5.2) 
Mathematics 23.3 (5.0) 18.6 (3.6) 
Reading 24.1 (5.8) 20.2 (5.4) 
Science 22.9 (4.5) 19.4 (4.0) 
ELA 23.2 (4.7) 19.1 (4.3) 
HSGPA 
Overall 3.53 (0.45) 3.09 (0.57) 


Note. Percentages may not sum to 100% due to rounding. 


Analyses 


Due to the nested structure of the data, various hierarchical logistic regression models were 
used to estimate students’ chances of succeeding in college as a function of ELA Benchmark 
attainment. To examine cumulative course performance, we used hierarchical logistic 
regression models to predict the probability of earning a 3.0 (equivalent to a B grade) or higher 
cumulative GPA at the end of year one through the end of year four as a function of whether 
the ELA Readiness Benchmark was met. Parameter estimates for both the intercept and slope 
of the Benchmark indicator (met/not met) were allowed to vary across institutions. Analyses of 
cumulative grades beyond year one were based on the subsample of students who remained 
enrolled in college. For students who were not enrolled during the second term of their first 
year, their GPAs from the first term were carried forward and used in the year one cumulative 
GPA analyses. 


For the retention/persistence outcome, hierarchical logistic regression models were used to 
estimate students’ chances of remaining enrolled in college. Students who were no longer 
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enrolled but who had completed a degree were categorized as being retained. Similar to the 
cumulative GPA outcome, both the intercept and the slope of the Benchmark indicator were 
allowed to vary across institutions. 


Hierarchical discrete-time survival models under the proportional hazards assumption 

were developed to predict degree completion from ELA Benchmark attainment (Singer & 
Willett, 1993; Reardon, Brennan, & Buka, 2002). This approach simultaneously models all 
time periods, while also accounting for censored observations due to the various freshman 
cohorts being tracked for differing lengths of time. In these models, the logit of the conditional 
probability of degree completion in a particular term, given that no degree was earned prior 

to that term, was modeled as a linear function of term indicators and the ELA Benchmark 
indicator. The discrete-time analyses focused on fall and spring terms. There were very few 
degrees given in the summer terms; summer term degree completion was therefore combined 
with that for the prior spring term. Term indicators for term 6 (spring/summer term of year three) 
through term 12 (spring/summer term of year six) were included in the bachelor’s degree 
completion models for the four-year sample. For the two-year sample where completion of 

an associate’s or bachelor’s degree was examined, term indicators for term 4 through 

term 12 were included in the models. Parameter estimates for the term indicators and the 
ELA Benchmark indicator were allowed to vary across institutions (the latter for the four-year 
sample only). 


The GLIMMIX procedure for generalized mixed models, available in SAS 9.2, with the Laplace 
estimation method was used to fit the models. The ELA readiness indicator was a statistically 
significant predictor for all outcomes across all time points, p < 0.0001, unless otherwise 
indicated in the tables of the parameter estimates provided in Appendix B. 


Results 


Cumulative GPA. Summarized in Table 9, the results indicate that the likelihood of earning 

a cumulative GPA of 3.0 or higher was strongly related to meeting the ACT ELA Benchmark 

of 20 at both four- and two-year institutions (refer to Table B3 in Appendix B for parameter 
estimates for the models). For example, for students at a four-year institution who met the 

ELA Benchmark, their chances of earning a cumulative GPA of 3.0 or higher in year one was 
57%. Their chances rose to a high of 69% in year four. In contrast, for students not meeting the 
ELA Benchmark, their chances of earning a cumulative GPA of 3.0 or higher across the four 
years was much lower, ranging from a low of 27% for year one to a high of 37% in year four. 


Table 9. Students’ Estimated Probability of Earning a Cumulative GPA of 3.0 
or higher by ELA Benchmark Attainment 


Year 1 Year 2 Year 3 Year 4 
N Pr N Pr N Pr N Pr 

Four-year institutions 

Not met 14,747 0.27 10,089 0.29 8,607 0.32 7,923 0.37 

Met 50,647 0.57 40,812 0.63 36,985 0.66 35,792 0.69 
Two-year institutions 

Not met 3,006 0.32 1,804 0.34 1,337 0.32 1,086 0.33 

Met 2,554 0.48 1,615 0.54 1,266 0.50 1,025 0.53 


Note. Cumulative GPA for year 2 and beyond is based only on students who remained enrolled in college. N = sample 


size; Pr = probability. 


Persistence. Similar to what was found for cumulative GPA, students who met the ELA 
Readiness Benchmark were more likely than those who did not meet the Benchmark to persist 
in college (see Figure 2; parameter estimates are provided in Table B4 of Appendix B). For the 
four-year sample, students’ chances of remaining enrolled in the initial institution attended in 
year two was 76% for those who met the ACT ELA Benchmark as compared to 65% for those 
who did not meet the ELA Benchmark. By year four, 56% of four-year students who met the 
Benchmark remained enrolled at their initial institution as compared to only 41% for those who 
did not meet the Benchmark. For the two-year sample, students’ chances of remaining enrolled 
in an in-state institution at year two were similar between those who met and those who did not 
meet the ACT ELA Benchmark. However, by year four, students from the two-year sample who 
met the Benchmark were more likely than those who did not to remain enrolled in college (44% 


vs. 36%). 
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Figure 2. Probability of remaining enrolled in college by ELA Benchmark attainment. 
Returning to the initial institution was examined for the four-year sample. Remaining 
enrolled in an in-state college was examined for the two-year sample to account for 

student transfer. 


Degree completion. Finally, the results indicate that students entering college academically 
ready for ELA-related coursework were more likely to complete a college degree than those 
who were not (see Figure 3; Table B5 of Appendix B includes the parameter estimates from the 
models). For the four-year sample, students’ chances of completing a bachelor’s degree within 
six years from their initial institution attended was greater than one-half (51%) for those who 
met the ELA Benchmark. In comparison, only about one-third (32%) of those who did not meet 
the Benchmark did so. For the two-year sample where students were tracked across in-state 
public institutions, students’ chances of earning an associate’s or bachelor’s degree within six 
years was 42% for those who met the Benchmark as compared to only 29% for those who did 
not met the ELA Benchmark. 
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Figure 3. Probability of completing a degree in four, five, or six years by ACT ELA 
Benchmark attainment. Bachelor’s degree completion primarily from the initial 
institution was examined for the four-year sample. Associate’s or bachelor’s degree 
completion within the state public higher education system was examined for the two- 
year sample.” 


In sum, the ELA Readiness Benchmark not only provides information to students about their 
likelihood of earning a grade of B or higher in typical first-year English and social science 
courses, but it is also related to a variety of important long-term academic outcomes such 
as earning a cumulative GPA of 3.0 or higher through the fourth year of college, persisting in 
college, and ultimately earning a degree. 


ACT ELA Benchmark Attainment for Recent ACT-Tested High 
School Graduates 

In the academic year of 2015-16, 519,922 students (25%) from the 2016 ACT-tested high 
school graduating class took the current ACT writing test and so they had an official ACT ELA 
score. Of these students, 71% met the ACT English Benchmark, 55% met the ACT Reading 
Benchmark, and 61% would be classified as meeting the ACT ELA Benchmark of 20 (see 
Table 10). For this subset of the 2016 ACT-tested high school graduating class, the percentage 
of students meeting the ACT ELA Benchmark varied across the racial/ethnic groups but not 
by gender (Figure 4). By race/ethnicity, the highest percentages for ACT ELA Benchmark 
attainment were seen among Asian and White students, while the lowest percentages were 
seen for African American and American Indian students. 


5 The degree completion rates between the four- and two-year samples are not directly comparable given that different 
definitions of degree completion were examined. 
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Table 10. ACT College Readiness Benchmark Attainment for 2016 ACT-Tested High 
School Graduates 


Percentage meeting ACT Benchmark 


Group of 2016 ACT-tested high 


school graduating class ACT English Benchmark ACT Reading Benchmark ACT ELA Benchmark 
Total class (N=2,090,342) 61% 44% -- 
rey ea al 71% 50% erm 
100 
90 
80 76 75 
iy 62 61 
® 60 56 
5 50 43 
2 40 32 33 
30 
20 
10 
0 
African American Asian Hispanic Pacific White Female Male 
American Indian Islander 
Race/ethnicity Gender 


Figure 4. ACT ELA Benchmark attainment by race/ethnicity and gender for 2016 
ACT-tested high school graduates who took the current ACT writing test in 2015-16. 


Given that the ACT ELA Benchmark was based on the same courses that were used to 
derive the ACT College Readiness Benchmarks in English and reading, ACT ELA Benchmark 
attainment (met; not met) was compared to ACT English and Reading Benchmark attainment 
(met both; met English only; met reading only; met neither) for the subset of 2016 ACT-tested 
high school graduates who took the current ACT writing test (Table 11). For the 2016 
ACT-tested high school graduating class, 88% of the students would be classified consistently 
on ELA, English, and reading readiness according to (1) the ACT ELA Benchmark of 20 and 
(2) the individual ACT College Readiness Benchmarks in English and reading of 18 and 22, 
respectively. For those classified consistently, 51% would be identified as being academically 
ready for typical first-year English and social science courses (cell shaded in dark blue in 
Table 11), while 37% would be identified as not being ready for those courses (cells shaded 
in light blue in Table 11). For the remaining 12% of students, 10% met the ELA Benchmark 
but did not meet at least one of the English and Reading Benchmarks and 2% did not meet 
the ELA Benchmark but met both the English and Reading Benchmarks. The disparities 
among students who met the ELA Benchmark but did not meet both the English and Reading 
Benchmarks was due primarily to students not earning an ACT reading score of 22 or higher. 
For the 2% who met both the English and Reading Benchmarks but did not meet the 

ELA Benchmark, it was due to lower ACT writing scores.” 


2° These students did not meet the ACT ELA Benchmark because of their relatively lower scores on the 
ACT writing test: 65% earned a 5 or below and 33% earned a 6 (out of a possible 12 points). 
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Table 11. Cross-tabulation of ACT ELA Benchmark Attainment by ACT English and 
Reading Benchmark Attainment for 2016 ACT-Tested High School Graduates who 
Took the Current ACT Writing Test 


ACT ELA Benchmark 


Met Not met 
ACT English and Reading Benchmarks N % N % 
Met both 9,683 1.9% 
Met English only 47,406 9.1% 50,660 9.7% 
Met reading only 4,257 0.8% 8,835 1.7% 
Met neither 2,763 0.5% 132,659 25.5% 


Note. Dark blue shaded cells highlight the scenario where there is consistency in meeting the two sets of ELA-related 
Benchmarks, while the light blue shaded cells highlight the scenarios where there is consistency in not meeting the two 
sets of ELA-related Benchmarks. 


Discussion 


The current study establishes a preliminary ACT ELA Readiness Benchmark to help identify 
students who have a high probability of succeeding in the first-year ELA-related courses 
typically taken by all college students regardless of their chosen academic major. Similar to 
what was done in the development of the ACT STEM Readiness Benchmark (Mattern et al., 
2015; Radunzel et al., 2015), the first study examined first-year, ELA-related course-taking 
patterns of students using actual course transcript data. The results indicated that the ELA- 
related first-year courses commonly taken by students were the same courses used in the 
establishment of the ACT College Readiness Benchmarks for the ACT English and reading 
tests (Allen, 2013) and that this was consistently seen across students’ major categories, 
including among students from more specific ELA-related majors. These courses were English 
Composition |, American History, Other History, Psychology, Sociology, Political Science, 

and Economics. So unlike results from the Mattern et al. (2015) study that found significant 
differences in the typical mathematics and science courses taken between STEM and non- 
STEM majors, this study did not find substantial differences across major categories in the 
typical English and social science courses taken during the first year. These findings suggest 
that there is less of an emphasis on the ELA Benchmark being associated with course success 
for specific major fields of study, unlike the case for the STEM Benchmark. These results are 
corroborated by other studies showing that well-developed literacy skills are important for all 
students to be able to successfully read and write across the college curriculum (Defazio, 
Jones, Tennant, & Hook, 2010; Thompson, 2011). 


The current study also found that based on students’ actual course grades, the ELA score 
needed to ensure a reasonable probability of success (50% for a B or higher) in first-year 
English and social science courses was estimated to be 20. We also evaluated the importance 
of meeting the ELA Readiness Benchmark for success beyond first-year ELA-related courses. 
The findings indicated that meeting the ELA Readiness Benchmark is associated with greater 
chances of earning a cumulative GPA of 3.0 or higher over time, persisting in college, and 
earning a college degree. 


Providing students with this type of information early on will help them determine their 
preparedness for success in a wide array of ELA-related courses typically taken during the first 


year of college. Hopefully, this will help students develop their literacy skills prior to entering 
college, thereby preventing the need to take remedial English and reading courses. Moreover, 
early grade readiness benchmarks can be linked to the ACT ELA Benchmark, providing an 
early signal to students on their ELA readiness so that there is time to intervene well before 
they graduate from high school. 


A limitation of the current study was that this preliminary Benchmark was based on estimated 
ELA scores that used concorded ACT writing scores. The data used to derive the ELA 
Benchmark was based on college outcomes of freshman cohorts who took the former 

ACT writing test prior to the introduction in 2015 of the current ACT writing test. For these 
students, ELA scores were estimated based on their ACT English, reading, and concorded 
writing scores, and then compared to student outcomes in specific first-year college courses. 
Supplemental analyses conducted to evaluate the use of concorded writing scores in 
estimating ELA scores suggested that this method is reasonable for the development of 

a preliminary ELA Benchmark (see Appendix A for further details). There are plans to re- 
evaluate the ELA Benchmark once sufficient college course-transcript data become available 
for students who took the current ACT writing test that includes freshman cohorts of 2016 and 
later. 


The representativeness of the samples used to develop and validate the ELA Readiness 
Benchmark is another limitation of the current study. Study 1 used data from just four states, 
and Studies 2 and 3 drew heavily upon institutions in the Southwest region. At the student 
level, the ACT writing test is optional, so not all ACT-tested students have all the scores 
needed to calculate an ELA score. Research suggests that students who take the ACT writing 
test tend to be better prepared academically than the average ACT-tested student (Table 10; 
ACT, 2015; ACT, 2016a). To mitigate these issues, both institutional and student weighting was 
employed to ensure that the sample was representative of the larger population of ACT-tested 
students nationally, and not only to the subset of students who took the ACT writing test. 


Clearly, both cognitive and non-cognitive factors contribute to academic and workplace 
performance (e.g., Allen & Robbins, 2010; Barrick & Mount, 1991; Le, Robbins, & Westrick, 
2014; Nye, Su, Rounds, & Drasgow, 2012; Poropat, 2009; Richardson, Abraham, & Bond, 
2012; Robbins, Lauver, Le, Davis, Langley, & Carlstrom, 2004; Van Iddekinge, Putka, & 
Campbell, 2011). For example, motivation and interest-major congruence have been shown to 
be predictive of timely degree completion after taking academic preparation into consideration 
(Allen & Robbins, 2010). Besides using the ACT ELA score, future research should include 
non-cognitive factors, high school coursework, and high school grades to help predict student 
success in ELA-related courses in the first year of college. That being said, the ELA Readiness 
Benchmark still provides valuable information for providing feedback on students’ literacy skills 
and is useful for tracking ELA proficiency rates across time and entities (e.g., schools, districts, 
states). 


The need for the integration of English, reading, and writing skills has become more widely 
recognized (CCSSI, 2010; Camara et al., 2015; Mattern et al., 2014), and the development 
of the preliminary ACT ELA Readiness Benchmark provides educators an estimate of the 
ELA skill level needed to succeed in seven ELA-related courses commonly taken by first- 
year college students. Previous research has demonstrated the difficulty of students catching 
up academically (Dougherty, 2014; Mattern et al., 2014), and with early-grade readiness 
benchmarks linked to the ACT ELA Benchmark, educators and parents will be able to see if 


23 


m ACT Research Report Development and Validation of a Preliminary ELA Readiness Benchmark based on the ACT ELA Score 


24 


students are on track as early as the third grade. This will allow time for educators, parents, 
and students to take action well before students approach high school graduation. Ideally, 
efforts to develop students’ literacy skills will lead to fewer college students needing remedial 
English and reading coursework, and, ultimately, will lead to enhanced overall performance in 
college courses and higher college graduation rates. 


For students who do not take the optional ACT writing test and hence are not provided with 
an ACT ELA score, students can use the ACT Benchmarks in English and reading to gauge 
their ELA readiness.?” A comparison between ACT ELA Benchmark attainment and 

ACT English and Reading Benchmark attainment (met both; met English only; met reading 
only; met neither) suggested that there is relatively high consistency in the message being 
provided on ELA readiness between these alternative definitions. That being said, the writing 
test does contribute unique information and together with the other two domains, provides a 
clear picture of a student’s strengths and weaknesses in ELA. 


Another important skill discussed in ACT’s ELA framework focuses on the ability to handle 
complex texts. To provide students with information on their proficiency in understanding 
complex texts, students who take the ACT reading test are also provided with an indicator for 
Understanding Complex Texts that assesses their ability to make global bridging inferences 
across a range of increasingly complex texts (Allen, Bolender, Fang, Li, & Thompson, 2016). 
Based on this indicator, students’ performance is categorized into the following three levels: 
below proficient, proficient, and above proficient. These categories were empirically derived 
based on students’ typical chances of being successful in college courses where more 
complex texts are often encountered such as American History, English Literature, Other 
History, Other Natural Science, Physics (without Calculus), Sociology, and Zoology. For more 
details about this other ELA-related measure, see the full report by Allen et al. (2016). 


In sum, given the exceeding importance of ELA skills in both education and the workplace 
(Camara et al., 2015; Mattern et al., 2014), this newly developed indicator of ELA readiness will 
provide useful diagnostic information to students, parents, and educators. The ultimate goal is 
that this information will be used to identify at-risk students and provide academic interventions 
in order to support all students in achieving educational and workplace success. 


7 Note that the ELA Benchmark of 20 also corresponds to the average of the ACT Benchmarks in English (18) and 
reading (22). 


Appendix A 


An Empirical Validation of the Use of Concorded Writing 
Scores in Estimating ELA Scores for Students with Former 
ACT Writing Test Scores 


ACT began reporting English Language Arts (ELA) scores in September 2015 when the 
current ACT writing test was first launched. While the former ACT writing test consisted of 

a 30-minute test using a holistic six-point rubric, the current ACT writing test was enhanced 
by redesigning the writing prompts, extending the testing time to 40 minutes, and using an 
analytical rubric for scoring on four writing domains. In addition, the score scale used for 
reporting the overall writing test score changed from the previous 2 to 12 scale to a 1 to 36 
scale. Beginning in September 2016, although the writing test (task) had not changed, 

ACT introduced a new way to report the overall writing test score as the rounded average of 
the four domain scores that ranges from 2 to 12 (ACT, 2016c). However, the 1 to 36 writing 
scale scores that were introduced in fall 2015 continue to be used in the calculation of 
students’ ACT ELA scores. The ACT ELA score is the rounded average of the English, reading, 
and writing scores; it ranges from 1 to 36. See ACT (2016b) for more information about the 
ACT writing test. 


To help with transitioning from the former to the current ACT writing test, a special study 
was conducted in fall 2014 to examine the relationships between scores on the former 

ACT writing test and scores on the current ACT writing test. Based on scores from 
approximately 800 students who took both a former ACT writing test prompt and a current 
ACT writing test prompt, a concordance table between the 2 to 12 scores of the former 

ACT writing test and the 1 to 36 scale scores of the current ACT writing test was developed 
and made available to test users (ACT, 2015). The concordance is not intended to be 

used for making high-stakes decisions for individual students. However, it may serve other 
purposes when the distributions of group level scores are considered. The intent of these 
supplemental analyses is to provide empirical support for some uses of the concordance table. 
For this study, the use of the concordance table is to estimate ELA scores for the purpose of 
developing a preliminary ELA Readiness Benchmark. 


As described in the body of the paper, the development of the preliminary ELA Readiness 
Benchmark was based on data from students who took the ACT writing test prior to 
September 2015 when ELA scores were not reported. ELA scores were estimated for these 
students using the concordance table between the former and current ACT writing scores. 

It was not known whether or how much the estimated ELA score distributions based on the 
concorded writing scores would differ from the actual ELA score distributions had the current 
ACT writing test been taken. It was also not known how using estimated ELA scores might 
affect the relationship between ELA scores and course grades. This appendix describes the 
supplemental analyses and corresponding results that provide empirical support for the use of 
the estimated ELA scores in large samples. 


Data and Method 

Matched samples were obtained from students who took the former ACT writing test in the 
2014-15 academic year and from those who took the current ACT writing test in the 2015-16 
academic year. First, a random sample was drawn from the pool of students who took the 
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writing test in 2014-15 to establish the English and reading bivariate score distribution for the 
sample. Then, from the pool of students who took the writing test in 2015-16, a frequency- 
matched sample was drawn to ensure that the 2014-15 and 2015-16 samples had the same 
English and reading bivariate score distribution. Next, ELA scores were estimated for students 
who took the former writing test using the concorded writing score values. Several different 
sample sizes were used for each of the following analyses to examine the extent that the 
sample size might affect the results. 


To examine how well the ELA score distribution can be estimated using concorded writing 
scores, the distribution of the estimated ELA scores for the sample of students who took 
the former writing test was compared to the distribution of actual ELA scores for the sample 
of students who took the current writing test. It was assumed that if the concordance table 
was not appropriate to use for the purposes of this study, then there would be detectable 
differences in the distributions of ELA scores between the two samples matched on the 
ACT English and reading bivariate distribution. The sample sizes used for these analyses 
included 50, 100, 500, 1,000, 2,000, and 5,000. 


To validate the use of estimated ELA scores in examining relationships between ELA scores 
and course grades, the relationship between high school grade point average (HSGPA; 
another academic performance outcome) and the estimated or actual ELA scores was 
examined for each sample. More specifically, the ELA score that was associated with at least 
a 50% chance of earning a specific HSGPA threshold was identified from a simple logistic 
ELA score/HSGPA regression model. HSGPA instead of first-year course grades was used as 
the outcome variable in these analyses. This was because college outcomes data were not 
available for students who took the current ACT writing test in the 2015-16 academic year. 
HSGPA has been found in prior research to be moderately correlated (0.47) with first-year 
college GPA (Westrick et al., 2015). The HSGPA thresholds included 3.0 or higher, 3.25 or 
higher, and 3.50 or higher. The models were run using sample sizes of 5,000, 10,000, and 
100,000 per sample. 


Results 


The cumulative relative frequency distributions of the estimated ELA scores of the 2014-15 
sample and the reported ELA score distributions of the 2015-16 sample are presented in 
Figure A1, with each graph presenting results for a different sample size (50, 100, 500, 1,000, 
2,000, and 5,000, respectively). Recall that these samples were frequency-matched based 

on the English and reading bivariate score distribution. As shown in Figure A1 and Table A1, 
although minor differences exist for small samples, the estimated ELA score distribution for 
students who took the former ACT writing test (using the concorded writing scores) was almost 
identical to the distribution obtained for the matched sample of students who took the current 
ACT writing test for large samples (1,000 or higher). 


Based on the results from the ELA score/HSGPA logistic regression models, the ELA cut 
scores associated with at least a 50% chance of attaining a specific HSGPA threshold were 
determined at the various threshold values and sample sizes for each sample. For each 
scenario, the estimated ELA cut score (based on the 2014—15 sample) was similar to or within 
one point of that for the actual ELA score (based on the 2015-16 sample). 


In conclusion, findings from these supplemental analyses provide evidence to support the 
use of estimated ELA scores in developing a preliminary ACT ELA Readiness Benchmark. 
However, it is important to note that ACT does not recommend the use of the concordance 
table for high stakes decisions for an individual student. 
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Figure A1. Comparison of distributions of the estimated ELA scores and the reported 
ELA scores for frequency-matched samples. 
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Table A1. Cumulative Frequency Distributions of Estimated Versus Actual 
ELA Scores from Matched Samples by Sample Size 


ACT N=50 N=100 N=500 N=1000 N=2000 N=5000 
ae Estimated Actual Estimated Actual Estimated Actual Estimated Actual Estimated Actual Estimated Actual 
1 0 0 0 0 0 0 0 0 0 0 0 0 
2 0 0 0 0 0 0 0 0 0 0 0 0 
3 0 0 0 0 0 0 0 0 0 0 0 0 
4 0 0 0 0 0 0 0 0 0 0 0 0 
5 0 0 0 0 0 0 0 0 0 0 0 0 
6 0 0 0 1 0 0 0 0 0 0) 0 0 
7 2 0 1 2 0 0 1 0 1 0 1 0 
8 2 0 1 2 2 1 2 1 1 1 2 1 
9 2 0 3 5 2 3 2 2 2 2 3 2 

10 2 0 7 6 4 4 4 3 3 3 4 4 
11 4 4 10 6 5 6 5 6 5 5 6 6 
12 4 4 10 9 7 8 8 9 8 8 8 9 
13 8 6 11 13 11 12 12 12 10 11 12 12 
14 10 12 14 14 14 15 16 17 14 16 15 17 
15 12 14 16 23 19 21 20 21 19 21 19 21 
16 18 20 21 24 23 25 25 26 24 25 24 25 
17 26 24 26 28 28 29 29 31 29 31 29 31 
18 30 32 33 36 35 33 35 37 35 37 35 37 
19 34 32 41 38 42 42 41 43 41 44 42 43 
20 40 34 45 42 49 48 48 49 48 50 48 49 
21 42 46 51 53 54 57 53 55 54 56 54 56 
22 50 52 59 61 61 64 59 60 60 61 60 61 
23 58 60 64 67 67 69 65 65 66 67 66 67 
24 62 66 73 74 72 75 72 71 71 73 72 73 
25 68 74 80 78 78 79 77 76 76 78 77 78 
26 70 78 86 83 83 83 82 82 81 83 82 82 
27 80 84 88 86 87 87 86 85 85 87 85 87 
28 86 92 90 89 90 91 90 90 90 90 89 90 
29 92 94 92 92 94 95 93 93 93 93 93 93 
30 94 98 94 93 95 97 95 96 95 96 95 96 
31 96 98 95 97 98 98 96 97 97 97 97 98 
32 100 100 95 98 99 99 98 99 98 99 99 99 
33 100 100 97 99 100 100 99 99 99 99 100 100 
34 100 100 100 99 100 100 100 100 100 100 100 100 
35 100 100 100 100 100 100 100 100 100 100 100 100 
36 100 100 100 100 100 100 100 100 100 100 100 100 
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Appendix B 


Table B1. Classification of General Major Categories 


General major category CIP codes 

Arts & Humanities 05, 16, 23, 38, 39, 50, 54 

Social & Behavioral Sciences 09, 42, 43, 44, 45 

General Studies 24 

Life Sciences 01, 03, 19, 26, 51 

Science & Engineering 11, 14, 27, 40 

Career Related 041, 10, 12, 15, 41, 46, 47, 48, 49 


Other Non-Science Related Majors 13, 22, 30, 31, 52 


Note. The two-digit CIP codes encompass all codes under them in the hierarchy unless otherwise denoted. 
For more information on specific CIP codes, go to the National Center for Education Statistics website at 
https://nces.ed.gov/ipeds/cipcode/Default.aspx?y=55 (National Center for Education Statistics, 2002; 2010). 


’ Excludes six-digit CIP codes 040800 (Architectural History and Criticism) and 040801 (Architectural History and 
Criticism, General) per NSF taxonomy of fields. These latter six-digit codes were included with Arts & Humanities. 


Table B2. Parameter Estimates from Hierarchical Logistic Regression Models 
for ELA Course Success 


B or higher C or higher 
Parameter Standard Parameter Standard 
Variable estimate error estimate error 
Intercept" -2.593 0.113 -0.672 0.129 
ACT ELA score’ 0.141 0.005 0.105 0.006 
eee 0 for English) “oe Gite a Ome 
Content indicator x ACT ELA score 0.051 0.006 0.064 0.008 


Note. ELA-related courses included English Composition |, American History, Other History, Psychology, Sociology, 
Political Science, and Economics. 


' The intercept and the slope for the ACT ELA score were allowed to vary across the course/institution combinations. The 
estimates (and standard errors) of the variability in the intercepts and slopes across the course/institution combinations 
were 1.760 (0.168) and 0.003 (0.0003) from the B or higher model and 2.300 (0.216) and 0.004 (0.0005) from the C or 
higher model, respectively. 
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Table B3. Parameter Estimates from Hierarchical Logistic Regression Models 
for Students’ Chances of Earning a 3.0 or Higher Cumulative GPA 


Mean/ ELA Benchmark 
Sample Year Variance Intercept (SE) Indicator (SE)' 
-1.013 1.287 
Mean 
{ (0.053) (0.048) 
; 0.101 0.064 
Variance 
(0.025) (0.019) 
-0.908 1.434 
Mean 
5 (0.050) (0.054) 
; 0.072 0.073 
Variance 
(0.021) (0.023) 
Four-year 
-0.739 1.424 
Mean 
‘ (0.051) (0.057) 
. 0.072 0.080 
Variance 
(0.021) (0.027) 
-0.541 1.351 
Mean 
4 (0.053) (0.058) 
. 0.076 0.077 
Variance 
(0.023) (0.028) 
-0.757 0.671 
Mean 
; (0.070) (0.100) 
. 0.087 0.147 
Variance 
(0.035) (0.086) 
-0.651 0.793 
Mean 
P (0.072) (0.087) 
. 0.056 0.021 
Variance 
(0.030) (0.039) 
Two-year 
-0.753 0.768 
Mean 
; (0.072) (0.097) 
. 0.028 0.015 
Variance 
(0.027) (0.037) 
-0.713 0.841 
Mean 
; (0.077) (0.113) 
: 0.027 0.057 
Variance 
(0.030) (0.069) 


Note: SE = standard error; Mean = estimated mean value of parameter across institutions; Variance = estimated variance 
of parameter across institutions. 


’ For each case, the parameter estimate for the ELA Benchmark indicator associated with the mean was significantly 
different from zero with p < 0.0001. 


Table B4. Parameter Estimates from Hierarchical Logistic Regression Models 
for Persistence 


Mean/ ELA Benchmark 
Sample/Outcome Year Variance Intercept (SE) Indicator (SE)' 
0.612 0.481 
Mean 
: (0.077) (0.038) 
. 0.268 0.031 
Variance 
(0.058) (0.012) 
-0.077 0.536 
Mean 
Four-year/ 3 (0.082) (0.042) 
Retention 0.300 0.050 
Variance 
(0.065) (0.017) 
-0.364 0.586 
Mean 
A (0.089) (0.044) 
, 0.360 0.054 
Variance 
(0.078) (0.018) 
0.592 0.1252 
Mean 
J (0.047) (0.081) 
, 0.018 0.069 
Variance 
(0.013) (0.060) 
-0.192 0.270° 
Mean 
Two-year/ 4 (0.043) (0.065) 
Persistence 0.011 0.024 
Variance 
(0.011) (0.029) 
-0.587 0.339 
Mean 
j (0.049) (0.072) 
. 0.021 0.040 
Variance 
(0.015) (0.037) 


Note. SE = standard error; Mean = estimated mean value of parameter across institutions; Variance = estimated variance 
of parameter across institutions. 


' For most cases, the parameter estimate for the ELA Benchmark indicator associated with the mean was significantly 
different from zero with p < 0.0001. The exceptions are denoted otherwise. 

2 The parameter estimate for the ELA Benchmark indicator associated with the mean for this outcome was not 
significantly different from zero (p > 0.05). 

3 The parameter estimate for the ELA Benchmark indicator associated with the mean for this outcome was significantly 
different from zero with p < 0.001. 
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Table B5. Parameter Estimates from Hierarchical Discrete-Time Survival Models for 
Degree Completion 


Four-year institutions Two-year institutions 
Variable Mean/Variance Estimate (SE) Estimate (SE) 
Mean --1 -2.251 (0.139) 
Term 4 
Variance --1 0.516 (0.147) 
Mean --1 -3.636 (0.130) 
Term 5 
Variance --1 0.188 (0.118) 
Mean -5.114 (0.126) -2.941 (0.070) 
Term 6 
Variance 0.450 (0.144) 0.011 (0.023) 
Mean -4.358 (0.094) -3.984 (0.124) 
Term 7 
Variance 0.272 (0.083) 0.061 (0.076) 
Mean -1.704 (0.124) -3.221 (0.086) 
Term 8 
Variance 0.712 (0.154) 0.023 (0.037) 
Mean -2.731 (0.086) -3.739 (0.165) 
Term 9 
Variance 0.267 (0.070) 0.035 (0.096) 
Mean -2.387 (0.091) -3.249 (0.122) 
Term 10 
Variance 0.321 (0.077) 0.030 (0.056) 
Mean -3.757 (0.111) -3.983 (0.171) 
Term 11 
Variance 0.209 (0.089) -? 
Mean -3.598 (0.120) -3.600 (0.145) 
Term 12 
Variance 0.244 (0.108) - 
Mean 0.651 (0.037) 0.427 (0.051) 
ELA Benchmark indicator? 
Variance 0.034 (0.011) -2 


Note. SE = standard error. The term variables can be interpreted as: Term 4 = Spring/Summer term of year 2, 
Term 5 = Fall term of year 3, through Term 12 = Spring/summer term of year 6; Mean = estimated mean value of 
parameter across institutions; Variance = estimated variance of parameter across institutions. 


1 These terms were not included in the model for the four-year sample because too few bachelor’s degrees were earned 
during these terms. 

2 These indicators did not vary across institutions in the model for the two-year sample. 

3 For each sample, the parameter estimate for the ELA Benchmark indicator associated with the mean was significantly 
different from zero with p < 0.0001. 
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